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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments filed 06/02/2009 have been fully considered but they are 
not persuasive. On Page 9 of 14, Applicant argues that there is no apparent reason to 
combine Zawlinski, Hoffberg, and Strubbe. Examiner respectfully disagrees. The 
combination of familiar elements according to known methods is likely to be obvious 
when it does no more than yield predictable results. Furthermore, if a person of 
ordinary skill can implement a predictable variation, 103 likely bars its patentability. 
Applicant also states that the Zawlinski reference lacks providing descriptive 
information for each of the television commercials so that an association of the 
user's reaction with such descriptive information can be defined. In response, 
Examiner would like to point to Col. 5 lines 34-43 of Zawlinski where it is taught that 
there exists a system for associating a plurality of semantic descriptors (descriptive 
information) with each of a plurality of emotional responses of a human (user's 
reaction) at a given moment over a preselected time period during the presentation of a 
stimulus (each television commercial). 

Regarding Hoffberg's teachings, Applicant argues that an association between the 
user's current mood and a current input stimulus is not taught. This is shown in Col. 65 
lines 50-55 [An example of an implicit input is an observation of a man-machine 
interaction, such as a video game. The manner in which a person plays a video game or 
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otherwise interacts with a machine may provide valuable data for determining a mood or 
preference]. 

As for the reason one of ordinary skill in the art would create an association between an 
emotional response and descriptive information related to a stimuli that produces the 
emotional response, Zawlinski discloses that the object of his invention is to aid in the 
field of research such that the data collected is used to make recommendations. [Col. 1 
lines 5-15 and Col.3 lines 20-22]. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claim 1-8, 10-30, 36, 39, and 40 rejected under 35 U.S.C. 103(a) as being 
unpatentable overZawilinski, U.S. Patent No. 5,676,138, in view of Hoffberg et al., U.S. 
Patent No. 6,400,996 in further view of Strubbe U.S. Patent No. 5,483,278. 

Regarding claim 1 and 40, Zawilinski discloses a multimedia system comprising 
an emotional response analyzer. The system further comprises sensors for 
sensing the user's physical reactions (i.e., gaze, galvanic skin response, etc.) to 
various stimuli, including commercials, wherein said stimuli are associated with 
recognizable emotional responses (i.e., surprise, anger, disgust, etc.). Moreover, 
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this information can be indexed and associated with whatever stimuli were being 
displayed at the time of said response. (Fig. 1 & 2; Abstract; Col. 1 , Ln. 6-47; Col. 
3, Ln. 39-Col. 4, Ln. 47; Col. 5, Ln. 34-43, 63-Col. 6, Ln. 26, 35-Col. 7, Ln. 5; Col. 
9, Ln. 50-64). 

But, Zawilinski fails to disclose a microphone picking up vocalizations made by 
the user. In an analogous art, however, Hoffberg discloses using biometric 
sensor data to create a dynamic user preference profile, wherein voice patterns 
of the user are detected and associated with a recognizable emotional response 
(i.e., moods; col. 116, 11. 3-14, col. 61, 11. 14-30, col. 65, U. 23-55; a 
microphone, or electro-acoustic transducer, is inherently present in where an 
acoustic signal (i.e., voice) is interpreted by an electrical device). Thus, it would 
have been obvious to one having ordinary skill in this art at the time of Applicant's 
invention to modify the at least one sensor of Zawilinski to include a microphone 
for picking up vocalizations, as taught by Hoffberg, to provide additional 
physiological data from which the user's preferences can be inferred. 
Hoffberg also teaches a memory device for storing the association between the 
programming description and sensed reaction as the viewer preference (see 
Hoffberg, col. 113, lines. 47-66). Furthermore, Hoffberg discloses of a 
recommender that is configured to recommend subsequent programs based on 
the viewer preferences (col. 65 lines 16-22; The system provides an intelligent, 
adaptive pattern recognition function in order to recommend programs to the user 
based on the user input, a past history of use, and a context of use.) Therefore it 
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would have been obvious to one of ordinary skill in the art to combine the 
teachings of Zawilinski and Hoffberg so that physiological data from which user's 
preference is inferred can be used to provide recommendations to the user 
based on viewer preferences. 

The combined teachings of Zawilinski and Hoffberg fail to properly teach of a 
a monitoring system that monitors subsequent programs which become available 
to be displayed and a system for notifying and/or presenting a subsequent 
program based on the viewer preference. In an analogous art, Strubbe discloses 
a recommendation system which uses collected information about users to 
recommend programs. In particular Strubbe teaches of a monitoring system that 
monitors subsequent programs which become available to be displayed and a 
system for notifying and/or presenting a subsequent program based on the 
viewer preference (see Strubbe Col 2 lines 41-47). Therefore it would have been 
obvious to one of ordinary skill in the art to combine the teachings of Strubbe with 
Zawilinski and Hoffberg to devise a recommender system capable of suggesting 
subsequent programs for the benefit of aiding in the selection of user preferred 
programming. 

Claims 34 and 39 are encompassed within the rejection of Claim 1 . Thus, it is 
analyzed and rejected as previously discussed. 
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As to claim 2, Zawilinski and Hoffberg together disclose the claimed memory 
device (see Hoffberg, col. 1 13, U. 47-66). 

As to claim 3, Zawilinski further discloses the system comprises a plurality of 
sensors. (Col. 6, Ln. 3-26). 

As to claim 4, Zawilinski further discloses the claimed sensor-signal receiver 
(rigA, item 18; col. 6, 11. 27-35). 

As to claim 5, Zawilinski further teaches the system can analyze various 
physiological data including heart rate, galvanic skin response, etc. (i.e., 
aggregation of signals; see Zawilinski as applied to claim 1, above). 

As to claims 6-8, Zawilinski and Hoffberg together disclose the use of a camera 
(video pattern recognition; Hoffberg, col. 1 16, 1 1 .3-14) and video recorder 
recording images captured by the camera to an image library for comparing to 
video images received from the camera (internal database; Hoffberg, col. 59, 1 1 . 
33-61). 

As to claim 10, Hoffberg, further discloses the system comprises an 
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environmental sensor for sensing a change in the viewing environment, as 
claimed (col. 127, U. 5-40). 

Regarding claim 1 1 , Zawilinski and Hoffberg together disclose the claimed 
subject matter. (See the rejection of claim 1; Hoffberg discloses determining 
when a program segment is being received that corresponds to a pre-selected 
viewer response previously associated with a physical condition status (i.e., 
user's mood; col. 116, 1 1 . 3-14, 33-49, col. 118, 11.27-43).) 

As to claim 12, Zawfilinski and Hoffberg together disclose monitoring a plurality of 
viewer physical conditions (Zawilinski, col. 6, 11 .3-26). 

As to claim 13, Zawilinski further teaches the system can plot various stimuli 
changes over time (i.e., changes in condition relative to a baseline level). (Col. 9, 
Ln. 12-20). 

As to claim 14, Zawilinski and Hoffberg together disclose the claimed subject 
matter. In particular, Hoffberg discloses the physical condition is body 
temperature (col. 61, n. 13-30). 



As to claim 15, Zawilinski further teaches the system can analyzed the heart rate 
of a user, (see rejection of claim 1 ). 
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As to claims 1 6 and 1 7, see the rejection of claims 6-8, above. 

As to Claim 18, Zawilinski and Hoffberg together disclose the claimed subject 
matter. In particular, Hoffberg discloses determining characteristics of a 
displayed programming segment, associating a viewer response corresponding 
to a physical condition with a viewer preference level, and applying the 
preference level to enhance program selection (col. 100, 1 1 . 2-51 , col. 1 16, 1 1 . 
33-49, col. 118, 11. 21-43). 

As to claim 19, Hoffberg further teaches providing a notification that specified 
future programming will contain at least one segment possessing the at least one 
distinguishing characteristic (col. 116, 1. 50 - col. 117, 1.5). 

As to claim 20, Hoffberg further teaches enhancing the program selection by 
inserting a segment possessing the distinguishing characteristic (col. 114, 11.12- 
23). 



As to claims 21 and 22, Hoffberg further discloses the characteristic is derived 
from EPG information provided with the programming, including text (col. 150, U. 
38-54). 
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As to claim 23, Hoffberg further discloses the claimed recorder for automatically 
recording the segment corresponding to a pre-selected viewer response (see 
portions of Hoffberg cited above). 

As to claim 24, Hoffberg further discloses extracting information related to the 
program segment that corresponds to pre-selected viewer response from the 
programming, and automatically displaying the information (col. 124, 11. 45-57). 

The limitations of claim 25 are encompassed within the portions of Zawilinski 
used to reject claim 1 . 

As to claim 26, Zawilinski further teaches the system can analyzed the galvanic 
skin response of a user. (Col. 6, Ln. 10-19). 

As to claims 27 and 28, Zawilinski further teaches the system is capable of 
measuring the gaze of the viewer and the electrical activity in muscles, which 
relates to negative/positive facial expressions (i.e., visually observable 
response). (Col. 4, Ln. 13-31 and citations of Claim 1). 

As to Claim 29, Zawilinski further teaches the system can track the direction of 
the user's gaze. (Col. 9, Ln. 50-64). 
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As to claim 30, Zawilinski further discloses the system is capable of analyzing the 
movement (i.e., changes in direction) of the viewers gaze and correlating the 
gaze direction with specific time periods (col. 9, 1 1 . 50-64, col. 4, II. 42-47). 

As to claim 36, Zawilinski and Hoffberg together disclose the method of claim 34, 
wherein an audible response is detected to infer the listener's mood. Official 
notice is taken that laughter is a well-known audible indication of mood. Thus, it 
would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the system of Zawilinski and Hoffberg to include listener 
laughter as an audible response, thereby providing an addition indication of the 
listener's mood. 

1 . Claims 31 and 32 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Zawilinski in view of Hoffberg in further view of Strubbe U.S. Patent No. 5,483,278 
applied to claim 27 above, and further in view of Black et al. (US Patent No. 5,774,591). 

Claim 31 recites the method of claim 27, wherein the visually observable 
response includes the furrowing of the viewer's brow. As discussed above, the 
combined systems of Zawilinski, Hoffberg and Strubbe render obvious all 
limitations of Claim 27, and Zawilmski further teaches the system can analyzed 
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the electrical impulses in the user's muscles (i.e., which can related to facial 
expressions), but fails to specifically recite analyzing eyebrow furrowing. 
However, within the same field of endeavor, Black et al disclose a similar system 
which analyzes changes in the viewer's eye brows. (Abstract; Col. 7, Ln. 1-45; 
Col. 26, Ln. 50-67; Col. 28, Ln. 28-62). Accordingly, it would have been obvious 
to one having ordinary skill in this art at the time of Applicant's invention to 
combine the systems of Zawilinski, Hoffberg, Strubbe and Black in order to 
provide a system capable of analyzing facial features, thereby affecting the 
operation of a computer system. 

The limitations of claim 32 would be obvious variants of the limitations of claim 
31 . Since Black et al allow for the analysis of user eyebrows, analyzing the depth 
of movement in said eyebrows would only be an obvious variant. Accordingly, it 
would have been obvious to one having ordinary skill in this art at the time of 
Applicant's invention to modify the combined systems of Zawilinski, Hoffberg, 
Strubbe, and Black in order to provide a system which analyzes the depth of 
furrows, thus allowing for an analysis of the level of a viewer facial response (i.e., 
level of anger, surprise, understanding, etc). 



2. Claims 33 and 38 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Zawilinski, in view of Hoffberg, as applied to claims 1 1 and 34 above, and further in 
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view of Bentolila etal. (US Pat. Pub. No. 2003/0101449). 

Zawilinski, Hoffberg, and Strubbe together disclose the limitations of claims 1 1 
and 34, but fail to discuss the limitations of claims 33 and 38. However, within the 
same field of endeavor, Bentolila et al disclose a similar system which utilizes a 
Hidden Markov technique. (Par. [0021]). Accordingly, it would have been obvious 
to one having ordinary skill in this art at the time of Applicant's invention to 
combine the systems of Zawilinski, Hoffberg and Bentolila in order to provide a 
system for user profile data prediction. 

3. Claim 37 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Zawilinski in view of Hoffberg in view of Strubbe, as discussed under claim 34, and 
further in view of Shinohara. (US Pat. Pub. No. 2003/0005431). 

Claim 37 recite the method of Claim 34, wherein the audibly observable 
response is the inflection (i.e., changes in pitch or tone) of a listener's 
vocalization, tending to indicate a question has been enunciated. As discussed 
above, the combined systems of Zawilinski and Hoffberg render obvious all 
limitations of Claim 34, but fail to specifically recite the limitations of Claim 37. 
However, within the same field of endeavor, Shinohara discloses a similar 
system which analyzes spectral data related to speech patterns, such as pitch, 
tone, frequency, etc., in order to identify a television viewer. (Par. [0026]). 
Accordingly, it would have been obvious to one having ordinary skill in this art at 
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the time of Applicant's invention to combine the systems of Zawilinski, Hoffberg, 
Strubbe and Shinohara in order to provide a system which is capable of 
analyzing various tones, pitches, etc. of a given voice. 

Conclusion 

4. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to SAHAR A. BAIG whose telephone number is (571)270- 
3005. The examiner can normally be reached on 4/5/9. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chris Kelley can be reached on 571-272-7331. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Christopher Kelley/ 
Supervisory Patent Examiner, Art 
Unit 2424 
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